Gödel Forest | Teaching Models to Evolve Through Data: When One Failure Becomes the Forest's Shared Experience

Gödel Forest | Teaching Models to Evolve Through Data: When One Failure Becomes the Forest's Shared Experience

If an AI agent designs its own training data, trains a model, checks the results, and then revises the data in response, can it keep improving through that loop?

This is the problem that data-centric recursive self-improvement (Data-Centric RSI) aims to solve. Instead of changing only prompts or agent workflows around a frozen base model, the agent continually explores new data and training strategies, uses the generated data to update the target model's parameters, and feeds the model's observed performance into the next round of exploration.

But there is a practical bottleneck: every data hypothesis must be validated by actually training and evaluating a model. Search too deeply and the agent may become trapped in the wrong direction. Explore too many directions at once and limited compute is spread too thin. Isolated agents repeat one another's mistakes, while agents that exchange complete execution traces become burdened by long logs and context.

Our paper Gödel Forest: Balancing Search Depth and Breadth for Data-Centric Recursive Self-Improvement proposes a different solution: organize Data-Centric RSI as a forest of search trees that share experience and co-evolve. Each agent preserves its own long-horizon search trajectory, while the forest exchanges lightweight lessons distilled from successes and failures through shared evolution memory.

The idea is simple: do not share every step; share the findings worth remembering. A dead end in one tree can immediately warn the others away from an unpromising path, while a breakthrough in one tree can quickly seed new branches across other parts of the data space.

One-Minute Summary

• One step beyond agent-harness optimization. Data-Centric RSI does more than optimize prompts or toolchains around a model. By autonomously designing data and training strategies, the agent directly drives updates to the target model's parameters.
• Turn a single search into a forest. Each agent maintains a persistent search tree that can branch and backtrack. Multiple agents explore different data strategies in parallel, balancing search depth with exploration breadth.
• Share experience, not accumulated logs. The system distills successes and failures experiences into structured lessons of roughly 100 tokens and propagates them asynchronously through a leaderboard and shared evolution memory.
• Gains on all six tasks. Across the six RSIBench-Data benchmarks, Gödel Forest improves on the single-agent baseline by an average absolute 10.70 percentage points and reduces end-to-end wall-clock time on five tasks.
• Collaboration is more than adding compute. Three independent parallel agents reach only 47%, while shared memory raises the score to 54%. The key gain comes from cross-tree knowledge coordination, not naive parallelism.

Figure 1. Overview of the Gödel Forest Method.

Data-Centric RSI Lets the Method for Studying Data Evolve

Conventional agent self-improvement usually optimizes prompts, tool-use rules, or external workflows while leaving the underlying model parameters unchanged. Data-Centric RSI takes another route: it turns the agent into an autonomous data researcher that proposes data hypotheses, generates training data, configures training, and tests those hypotheses against the trained model's actual performance on downstream tasks.

A complete cycle can be summarized as follows:

1.Using the accumulated experimental history, the agent proposes a new data and training strategy.

2.Starting from fixed base-model parameters, it trains a candidate model on data generated by that strategy.

3.It evaluates the candidate model on downstream tasks.

4.It analyzes successes, failures, and behavioral defects, then writes that evidence back into the persistent search state.

5.It uses the evidence to refine the current direction, backtrack, or change course.

Here, recursion does not mean that a model being trained arbitrarily rewrites its own weights. Instead, empirical results from earlier candidate models continually change the agent's later data decisions. Training updates the candidate model's parameters, while the data-research agent updates its search direction through successive rounds of empirical feedback.

This makes Data-Centric RSI resemble automated scientific research. Every data recipe is a hypothesis, every training run is an expensive experiment, and the evaluation result determines whether a direction should be pursued, revised, or abandoned.

The Hard Part Is Preserving Both Depth and Breadth

The search space for data strategies is enormous. An agent must decide not only where samples come from, but also the task distribution, data format, supervision positions, trajectory truncation method, mixture proportions, and training configuration. More importantly, these choices often cannot be judged through static inspection. The model must be trained and its behavior observed.

A single-agent search can accumulate continuous context and pursue one direction for many rounds, but early choices can trap it in a narrow, path-dependent trajectory. Adding more agents broadens exploration, yet isolated agents repeat strategies that peers have already shown to fail. Broadcasting complete data and execution traces creates severe context bloat, making coordination rigid and expensive.

This creates the central dilemma of Data-Centric RSI:

How can a system preserve deep, continuous, long-horizon search while also exploring many directions under a limited training budget?

Gödel Forest does not choose between depth and breadth. It assigns depth to local search trees and breadth to the forest as a whole.

One Tree Goes Deep While the Forest Expands the Frontier

In Gödel Forest, each worker agent maintains a private, persistent search tree. Nodes represent data-engineering strategies that have already been tried, and parent-child links record how each strategy evolved. The agent does not have to advance from only the latest result. It can deepen a promising branch or return to an earlier node and explore a different direction.

Each search iteration has four stages:

•Diagnose. The worker analyzes specific failure modes from the previous candidate, such as runaway completion, ungrounded tool behavior, or skipped verification before submission.

•Propose. Using both its private experimental history and the forest's shared lessons, the worker selects a previous node and generates a new data-synthesis and training program.

•Validate and Evaluate. The worker first screens the candidate model on a lightweight diagnostic subset. Only candidates that pass the gating threshold proceed to full evaluation, pruning clearly unviable recipes early.

•Review. The worker records the experimental result and decides whether to exploit the current direction, revise the configuration, or backtrack to an ancestor node and open a new branch.

This persistent-tree structure gives every agent room for long-horizon research. One failure does not erase earlier high-performing models, and an aggressive new idea does not force the system to abandon a node that has already been validated.

How the Forest Communicates Through a Leaderboard and Experience Capsules

The trees coordinate through asynchronous shared evolution memory with two components:

•Global leaderboard. It records every fully evaluated candidate and its result, allowing each agent to see the current collective frontier.

•Structured empirical lessons. Each important experiment is compressed into what hypothesis was tested, how performance changed, why it changed, and what rule should guide the next step.

A lesson requires only about 100 tokens, far less than the raw data, complete trajectory, or execution log. Before a new search round, an agent retrieves lessons relevant to its current problem. After the experiment, it publishes any new finding with general value to shared memory. No agent needs to wait for another, and there is no global synchronization barrier.

Negative lessons become inductive guardrails that steer other trees away from confirmed failures. Positive lessons become catalytic seeds that encourage agents to test and extend the same idea with different data sources or training settings. Shared memory carries research leads that can continue to evolve, not answers to be copied mechanically.

All Six Benchmarks Improve by an Average of 10.70 Percentage Points

We evaluate Gödel Forest on RSIBench-Data across six challenging benchmarks covering repository-level software engineering, long-horizon terminal interaction, scientific reasoning, and competition mathematics. Every candidate starts from the same Qwen3.5-35B-A3B-Base model and is trained with LoRA SFT. Each search is constrained by a 16-hour wall-clock limit and a cumulative $500 Tinker budget.

Gödel Forest outperforms the single-agent baseline on all six tasks, with an average absolute gain of 10.70 percentage points. SWE-bench Verified rises from 35% to 54%, the harder SWE-bench Pro rises from 4% to 15%, and Terminal-Bench 2.0 rises from 5.62% to 17.98%. The gains are not confined to a single task format.

More exploration does not necessarily mean a longer wait. Through asynchronous parallel execution and rapid propagation of negative findings, Gödel Forest reduces end-to-end wall-clock time on five of the six tasks. SWE-bench Multilingual, for example, falls from 14.20 hours to 11.02 hours, while SWE-bench Pro falls from 9.54 hours to 7.49 hours.

Cumulative expenditure must be interpreted alongside task behavior. On some long-horizon agentic benchmarks, Gödel Forest costs more because stronger candidate models sustain multi-turn reasoning and tool use, while weaker baseline models may crash, loop, or terminate earlier. Our paper therefore reports score, wall-clock time, and cumulative cost together instead of presenting any single resource metric as a universal advantage.

Three Agents That Communicate Beat Three Agents Working Alone

A common question about multi-agent methods is whether the improvement comes from better coordination or merely from more parallel compute. On SWE-bench Verified, we vary the number of workers and compare the system with three independent workers that do not share information.

Figure 2. Ablation of Worker Count and Shared Memory

As the worker count increases from one to three, the official score rises from 45% to 51% and then 54%. Increasing the count to four leaves the score at 54% while substantially increasing wall-clock time and cost. Under a fixed total budget, excessive concurrency fragments compute across too many branches and reduces the depth available to each tree. In this setting, three workers provide the best balance of depth, breadth, and resource expenditure.

The more important comparison concerns shared memory. With a similar budget, three independent workers reach only 47%, 7 percentage points below the complete Gödel Forest system. Without shared lessons, workers revisit the same dead ends. With the leaderboard and structured empirical lessons, every failure and breakthrough becomes a shared asset for the entire system.

Gödel Forest is therefore not about simply running more agents. Its central mechanism turns parallel exploration into cumulative, transferable collective cognition.

A Real Collaboration From Runaway Generation to Testing Before Submission

A case study on SWE-bench Verified shows how shared lessons move across workers.

Figure 3. Cross-Worker Knowledge Synthesis on SWE-bench Verified

Worker 1 and Worker 3 initially used a data strategy that supervised all assistant turns. Both independently encountered the same problem: after completing the task, the model continued generating content. Their diagnostic scores were 0/9 and 0/10.

Worker 1 then restricted the training loss to the final assistant turn, restoring stable stopping boundaries and reaching 46% on the full evaluation. The system distilled this finding into its first shared lesson: masking the loss to the final assistant turn restores stable task termination. Worker 3 retrieved the lesson and transferred the same principle to its own independent data branch, raising its score from near zero to 22% and avoiding further training spend on a known failure mode.

The new strategy exposed a second problem. When only the final response carried training loss, intermediate Bash test commands received no direct supervision. The model stopped reliably but no longer performed pre-submission verification consistently. Worker 2 observed this after adding 22 real bug-fixing trajectories and distilled a second lesson: critical intermediate actions require explicit supervision.

To preserve stable termination while restoring active verification, Worker 2 converted intermediate test calls into pseudo-terminal targets through prefix-truncated trajectories. The model began running tests before submission, the candidate score reached 55%, and the checkpoint was selected for submission. The same checkpoint scored 54% in the official evaluation.

This case captures Gödel Forest's core operating pattern. One agent does not copy an answer to another. Instead, multiple agents jointly discover a phenomenon, form an explanation, distill a rule, transfer and validate it, and continue improving the strategy.

Scalable Self-Improvement Requires Knowledge to Flow Across Searches

Data-Centric RSI provides a direct path to model self-improvement. The agent does more than adjust the shell around a model; by designing training data, it changes the target model itself. But while training remains inside the search loop, every hypothesis is still expensive to validate, and the tension among depth, breadth, and resources does not disappear.

Gödel Forest separates individual search from collective learning. Persistent search trees protect long-horizon exploration, multiple workers expand directional breadth, diagnostic gating avoids unnecessary full evaluations, and compact shared evolution memory turns local experiments into globally reusable knowledge.

It also offers a broader lesson: more agents do not automatically make a stronger system, and more attempts do not automatically produce better self-improvement. What matters is whether the system remembers why a strategy worked, understands why another failed, and lets those lessons change the next search decision.

A single tree can search deeply. A forest that communicates can keep evolving.

---

Paper: Gödel Forest: Balancing Search Depth and Breadth for Data-Centric Recursive Self-Improvement https://arxiv.org/abs/2609.36675